Embodiments generally relate to camera localization. More particularly, embodiments relate to multi-modal real-time camera localization and environment mapping.
Virtual reality applications and three-dimensional (3D) gaming applications may submerse a user wearing a head-mounted camera into a 3D virtual environment. In such systems, camera localization and environment mapping may be conducted in order to translate “real world” movements of the user into movements in the 3D virtual environment. Conventional camera localization systems, however, may rely on only a single mode of data collection and analysis (e.g., collection and analysis of only combined depth and color data), which may result in failures that interrupt the user experience.